1. Field of the Invention
The present invention relates generally to natural language understanding and more specifically to knowledge-based methods for determining which of all possible meanings is intended by the speaker or writer of any portion of text in English or another natural language. As both words and phrases can often have multiple meanings, the problem is significant and many prior approaches have failed to yield satisfactory results. In order to achieve a high level of accuracy in natural language meaning interpretation, a knowledge base of information similar in breadth and depth to the knowledge of a human being is needed. Using digital or other automated strategies to represent and process this quantity of knowledge must be highly optimized to achieve satisfactory performance.
2. Description of Related Art
Under the general field of automated data processing, natural language processing falls under the specific category of artificial intelligence or, more recently, knowledge-based systems. Over the course of time, many of the trends and new developments in artificial intelligence have lead to new approaches for natural language processing. One such trend is “rules-based” systems applying production rules to perform forward-chaining or backward-chaining inference. The typical failing of these systems has been that the number of rules required to express the complexity of language has been extremely high and the sequential processing of these rules has lead to unacceptably long processing times.
An earlier trend involved list-based approaches using the LISP programming language. LISP provides an elegant computational mechanism of lists for processing many rules as constraints for interpreting text. Unfortunately, as the lists proliferate, list management becomes impractical and correlation of related constraints in different lists or even at opposite ends of the same list consumed too much computational effort and took too long. Those efforts were followed by logic-based approaches using the PROLOG programming language. This was a special syntax or structure for production rule type systems as described above and performance problems made these approaches equally impractical.
Some developers have used neural network approaches with connectionist and other neural networks to attempt to understand language. The model is appealing because the formulas, algorithms, and structures seem to mimic the brain. Neural networks have proven very good for two-dimensional and some three-dimensional problems such as automated character recognition and voice recognition. Neural networks appear to be good representations of the parts of the brain responsible for these tasks with limited dimensionality. Unfortunately, they require an inordinate amount of training time to “learn” about each new object. Furthermore, language understanding involves multiple dimensions, and the parts of the brain responsible for these functions appear to have more complex structures and processes. Therefore, a new approach is needed.
Most language analysis approaches, including those described above, have used different parsing techniques where the sentence is scanned from left to right, or right to left, and the syntactic structure determined based on patterns or rules. Newer approaches have added semantic constraint resolution to the syntactic analysis to improve the accuracy and usefulness of the results. These have introduced incremental advances in capabilities, but without the full analysis of context, they have not significantly increased the accuracy of language interpretation. Syntax is an important dimension of language understanding, and adding the dimension of semantics is important. But without the dimension of context, the accuracy will never approach human competency in language comprehension.
There have been various approaches that use genetic or evolutionary algorithms, semantic networks, conceptual schema and, most recently, Bayesian networks that seek to add the dimension of context. Though these approaches hold the promise of much higher accuracy, the proliferation of nodes in the semantic and Bayesian networks have made it impossible to achieve sufficient speeds to process language without unacceptable delays. Specifically, as these networks increase in size beyond dozens or hundreds of nodes, the processing times required to perform language analysis increases exponentially. When they attempt to include syntactic, semantic, and contextual information in these networks, the combinatorial explosion of processing time renders these systems useless.
3. Definitions of Terms
Activation
The spread of positive and negative electrical potential in the brain from neuron to neuron is called activation. Positive activation is called excitation and negative activation is called inhibition. In the multi-dimensional interpreter's knowledge network activation spreads from node to node based on associative links. This activation is a means of stochastic or fuzzy recognition used in the genetic selection algorithm.
Activation Wave
This expression represents the serial flow of excitation and/or inhibition triggered by a single input in a natural or artificial neural network. Natural and artificial neural networks can exhibit directional or chaotic flow of activation. An example of directional activation flow in a natural system is the human visual cortex which has multiple layers and through which activation flows sequentially from the back to the front, then on into the correlation and interpretation centers of the brain. Consequently, the deconstruction of the image in the brain's visual center is an output of a relatively directional wave of activation flow.
Once in the correlation and interpretation centers, the flow becomes much less directional or more chaotic. Activation flows in parallel to many specialized areas of the brain. These processing centers respond by sending back activation patterns that contribute to the emergent phenomena of recognition and interpretation that go on to support all cognitive functions. Whether directional or not, the path of any activation flow in a neural system can be traced backward from the point (neuron or node) where the flow stops to the point where it began, no matter how much it spreads or branches out in the process. The collection of all such serial paths triggered by a single input constitutes what we call a wave.
The complexity of the input may be arbitrary, but the more complex the input, the more complex the wave will be, hence the more difficult to trace. The science of tracing activation waves in the brain is not yet mature enough to trace the entire path of activation flow either backward or forward from neuron to neuron for a given input. Artificial systems, however, can be traced. Mimicking human activation flow patterns is one of the key objectives of many artificial neural systems.
Consciousness
In humans, consciousness is an emergent cognitive phenomenon usually active whenever one or more of the senses is perceptually active. Other cognitive phenomena, such as attention, derive from consciousness and may be described as “heightened states of consciousness”. In the multi-dimensional interpreter, consciousness is a state of accepting and processing input while maintaining a broader map of the spatial, temporal, commercial, and social context associated with its primary user.
Context
Context is a snapshot of the universe from a specific point of view to a specific depth. If the viewpoint is that of an astronomer at work, it could begin at her desk and include a radius of many thousands of light years. If the viewpoint is that of an electron in an inert substance, the context would encompass a very small distance. Context includes locations in space, points in time, activities, ideas, intentions, communications, motion, change, stasis, and any describable thing associated with the person, place, or thing to which the context applies.
Counterfactual
A counterfactual is a proposition that states that a certain associative proposition or causal link is unlikely, thus it spreads negative or inhibitory activation.
Disambiguation
Disambiguation is the process of resolving ambiguity, especially in words, symbols, or phrases that carry multiple possible meanings (polysemy). This is necessary for accurate interpretation of input human language text or utterances. Context is needed to disambiguate polysemous input.
Doping
In a genetic algorithm, doping is the process of introducing random or quasi-random variables into the equation, population or gene pool to affect the process, the output, or both. In this context quasi-random may mean; based on a random number generator, based on random selection of targets to which to apply variables, or based on non-random variables applied in a non-random way, but in which the variables have no describable association with the core interpretive processes or the targets to which the variables are applied.
Emergence
The term “emergent behavior” has been applied to the human brain and other complex systems whose internal behavior involves non-deterministic functionality or is so complex, or involves the interaction of so many pieces, that tracing the process from beginning to end is not possible or not feasible. Because of the power of computers and their ability to track internal activity, it is not possible to produce 100% untraceable processes just as it is not possible to produce a random number generator that is actually random and not ostensibly random. Information is said to “emerge” because it cannot be traced back to its source. In the context of the multi-dimensional interpreter, emergence is a computational behavior that mimics the inventor's understanding of the behavior of the human brain processes used to interpret human language.
Encoding Scheme
A way of representing something using a tightly specified symbol or object set. ASCII and EBCDIC are encoding schemes for symbol systems for alphabetic and numeric symbols. In this document, encoding scheme refers to a specific design for structuring language knowledge facts and real-world knowledge facts in the form of words and other human and machine readable symbols into conceptually relevant associations that can support automated or computerized access and processing. Human languages are such encoding schemes, but their irregularities make it difficult for use in their normal form for automated processing. Well-formed syllogisms or other logical statements with a finite set of connectors and operators are a more regular encoding scheme for knowledge of facts.
Expectation
Expectation is a concept that is relatively foreign to computing but essential to achieving high accuracy in natural language interpretation. Expectation is an a-priori set of contextual markers that describe a world familiar to the multi-dimensional interpreter system based on the world familiar to the human user. The more the system knows about the primary users and their surroundings, the better it will be able to determine the users' intentions based on the words they submit to the system.
Fitness
Fitness is a characteristic of a candidate solution or a part thereof. In a genetic selection process, survival-of-the-fittest is used to differentiate possible solutions and enable the one or more fittest solutions to emerge victorious.
Genetic Selection
Genetic Selection is a process of survival-of-the-fittest in which fitness algorithms are applied to multiple possible solutions and only the best survive each generation. Unlike winner-take-all processes in which only the single best candidate solution emerges, genetic selection can yield multiple surviving solutions in each generation. Then, as successive generations are processed, survivors from previous generations may die off if the succeeding generations are more fit.
Inference
For the purpose of this invention inference involves correlating multiple constraints contained in input and derived from other sources, and drawing conclusions based on testing the relative truth of multiple propositions affecting each constraint. Inference is what humans constantly do with our brains. Based on perceptions, humans make inferences about meaning, about the state of things, about consequences of actions, and about life, the universe, and everything. Inference involves applying logic to new information derived from our senses and remembered information stored somehow in the brain to form conclusions. Forming conclusions is important because the conclusions form a basis for correct interpretation and appropriate further processing. A characteristic of the conclusions drawn by the multi-dimensional interpreter is that it can abandon one quickly if newer information prompts it to do so.
Knowledge
For the purpose of this invention, the term knowledge means correlated information. This definition is part of a larger continuum in which everything interpretable can be assigned to some point on the continuum. The position of knowledge in this continuum can be described in terms of its complexity relative to other things in the environment that are interpretable. The level of facts that we can learn and describe in simple phrases is called existential knowledge or data. Data is the kind of knowledge expressed in almanacs. At the complex end of the knowledge continuum is one or more levels of meta-knowledge or knowledge about knowledge.
The term “noise” is borrowed from radio wave theory to describe interpretable things that interfere with the interpretation of or acquisition of knowledge. Noise, the simplest of all interpretable things, is made up of things in the perceptual environment or input that are less meaningful than data. An interpretation system must be able to process noise because it is omnipresent. Thus, a system must have knowledge that enables it to sort out the noise from the salient data, though this may be more of an attention function than actual knowledge. Once the noise in the environment is filtered out, all that remains is data, which can be correlated to constitute knowledge.
Data elements that humans process are input in the form of perceptual stimuli to the five senses. The specific types of data available are tactile sensations, tastes, smells, sounds and images. These perceptual inputs are processed in specialized areas of the brain, correlated in parallel, then used as the basis for cognitive processing. The multi-dimensional interpreter algorithms are primarily designed to interpret human language, but are also able to be generalized to interpret the other forms of sensory input described above.
Knowledge Base
The multi-dimensional interpreter uses a knowledge base that contains the basic information about things in the world and the way they are interrelated.
Knowledge Network
A massively interconnected network of information about how linguistic and real-world objects relate to one another.
Lexicon
Lexicon is a list of words, letters, numbers, and/or phrases used in a natural language, such as English, that express meaning or facts or represent objects or phenomena. The lexicon consists of a list of lexical items or objects, each a word or symbol or combination thereof. In the multi-dimensional interpreter the lexicon is part of the Knowledge Base.
Multi-Dimensional Processing
In this document, multi-dimensional processing refers to an approach of using an automated device or computer to analyze a string of input data, such as a sentence in English, from more than one perspective. Perspectives or dimensions include, for example, grammar or syntax, semantics, pragmatics, context, and morphology.
Mutation
A genetic algorithm mutation is a process of altering the characteristics of a candidate solution or a part thereof during the course of processing. The mutated result then can compete with other results for fitness as a solution.
Natural Language Processing
Natural language processing means using computers to analyze natural language input, such as English sentences, for the purpose of interpretation, paraphrasing, or translation.
Neural
Of, resembling, or having to do with the processing components and functions of the brain and/or its cells. Perceptual, inquisitive, communicative, interpretive, creative, and decisive cognitive processes occur in the brain through the functioning of its network of neuron cells. Those processes are neural, and automated processes designed to resemble the structure and/or functions of these processes are often characterized as neural.
Object
An object is a discreet string of one or more symbols or characters that has a beginning and an end and an unchanging content. If the content were to change through the addition, subtraction, or modification of one or more of its characters, it would become a different object.
Object Base
A set of lists of objects in which each list contains a collection of like objects. One such list is the lexicon whose constituent objects are all lexical items. Another such list is the sound base, which contains a list of sounds as machine-readable representations such as .wav files. The object base may also contain an image base holding a collection of images such as .gif and .jpeg files. Other lists, such as video, aroma, and tactile object lists can also be stored as object bases. The object bases are part of the knowledge base.
Polysemy
The linguistic phenomenon of multiple meanings applying to a single word, symbol, or phrase.
Real-World Knowledge
Real world knowledge includes facts about phenomena and things. In this document, real-world knowledge refers to information or data associated with other information or data in a meaningful or expressive way to represent facts in the world. Some facts describe the hierarchical relations between classes and subclasses of objects in the real world such as “a dog is Canine in the animal kingdom”. Other facts describe causal relations such as “gravity pulls physical objects toward each other”, and yet others describe constructive relations such as “a knob is part of a door”.
Stochastic
Non-deterministic or “fuzzy” processing techniques and encoding approaches that deliver output from a process that is virtually impossible to determine based on the inputs because of the sheer number of permutations and/or the complexity of the weighting mechanisms and processes to adjust weights during the course of the process and prior to the output.